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Teachers, Administrators, and Students: Rising Together 


Nothing we can do for our students matters more than giving them effective teachers. Research 
has proven this time and again. We need to do everything we can to give all our teachers the 
support they need to do their best work, because when they succeed, our students succeed. 
Without effective evaluation systems, we can't identify and retain excellent teachers, provide 
useful feedback and support to help teachers improve, or intervene when teachers consistently 
perform poorly. 

Indiana's irmovative pilot of evaluation systems emphasizes collaboration between teachers and 
administrators, who are working together with one goal in mind: helping students learn. This 
requires more than just careful design and strategic processes - it requires a sustained and 
meaningful culture shift at the school level. At the core of this culture shift is a change in the 
way we envision the relationship between teachers and administrators. Indiana's pilot 
evaluation project is informed by three core principles: 


Teachers deserve to 
be treated like 
professionals. 


Current evaluations treat teachers like interchangeable parts — 
rating nearly all teachers good or great and failing to give 
teachers the accurate, useful feedback they need to do their best 
work in the classroom. Teachers deserve regular feedback on 
their performance, opportunities for professional growth, and 
recognition when they do exceptional work. 


Better evaluations 
give principals the 
tools they need to 
become instructional 
leaders. 


The new systems will help principals support their teachers by 
helping them accurately pinpoint teachers' strengths and 
weaknesses. Helping teachers reach their potential in the 
classroom is a principal's most important job as an instructional 
leader, and a new evaluation system will hold principals 
accountable for helping all their teachers learn and grow. 


When teachers grow. Novice and veteran teachers alike can look forward to detailed, 
students grow. constructive feedback, tailored to the individual needs of their 

students. Teachers and principals will meet regularly to discuss 
successes and areas for improvement and set professional goals. 
The end result is better instruction for every student. 
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Legislative Overview 


If we want to dramatically improve education in Indiana, we must re-imagine the systems and 
policies that collectively shape the learning experience for students. 

In the spring of 2011, the Indiana legislature passed IC 20-28-11.5, a new law relating to the 
evaluation of all certified teaching staff and administrators. Prior to this legislation, evaluation 
systems around the state varied greatly in quality and consistency. The new law introduced 
three main requirements of all evaluation systems: 

• Every teacher must receive an evaluation annually; 

• Every evaluation system must include four performance categories: Highly Effective, 
Effective, Improvement Necessary, and Ineffective; and 

• Every evaluation system must incorporate measures of student growth and achievement 
as a significant portion of a teacher's evaluation. 

The new legislation positioned Indiana at the forefront of a nationwide push to elevate the 
teaching profession by giving educators regular, meaningful feedback on their job performance. 
All corporations must include the three legislative requirements in their evaluation plans by 
start of school year 2012-2013, or as soon as current contractual obligations expire. Each 
corporation must submit an evaluation plan to the Indiana Department of Education (IDOE), 
which will be made publicly available online to serve as a resource for all corporations in the 
state. 

The Indiana Evaluation Pilot 

To support corporations in this work, the IDOE created a model teacher evaluation system. 
This system, named RISE, was developed over the course of two years by the Indiana Teacher 
Evaluation Cabinet. The Cabinet is composed of a diverse group of educators and 
administrators from around the state, more than half of whom have won awards for excellence 
in teaching. RISE is an optional, modifiable evaluation system. Each school corporation is 
different, and corporations may choose to adopt RISE in full, adopt RISE with modifications, or 
develop their own system completely independent of RISE that fits the imique needs of their 
principals, teachers, and students. 

In school year 2011-2012, the DOE, with the support of TNTP', is piloting RISE and following 
the implementation of other locally developed or purchased evaluation systems in order to test 
and gather feedback to share statewide. The pilot presents an opportunity for school 
corporations to gain hands on experience with the new process, recognize their best educators, 
and to make sure that every classroom is led by a great teacher. Additionally, it offers the IDOE 
a valuable opportunity to gain feedback and insight from the field, which can be used to 
sharpen and refine RISE for the following year. 
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Six corporations out of 37 applicants were chosen for the pilot based on diversity of location and 
size as well as readiness to implement, for the pilot evaluation year. Three of these 
corporations— Greensburg Community Schools, Fort Wayne Community Schools, and 
Bloomfield School District— are piloting the RISE evaluation model. The other three 
corporations— Beech Grove City Schools, MSD Warren Township, and Bremen Public Schools — 
are piloting alternative evaluation systems. For a profile of each of these pilot districts and 
further details on evaluation model design, please see Appendix A. 

This report will summarize lessons from the pilot thus far, with a particular focus on planning, 
design, and initial implementation. The findings and recommendations are intended to guide 
corporations in developing evaluation plans for implementation in 2012-2013. This report is not, 
however, a comprehensive planning or implementation guide. For additional guidance on 
formulating an evaluation plan and other topics mentioned in this report, please visit the DOE 
guidance website. 

A follow-up report released this summer will provide detailed conclusions and further 
recommendations from the pilot year. 

Interim Findings 

The following interim findings and recommendations are based on multiple sources of data 
collected throughout the first half of the pilot year. Pilot corporation teachers and evaluators 
were surveyed twice, once in August and again in January, to understand their experience with 
prior evaluation systems and the new system this year. Additionally, qualitative data was 
collected via individual interviews and focus groups. 

Planning and System Design 

1. Designing an evaluation system requires significant time and capacity. 

Pilot corporations faced significant time and capacity challenges when designing evaluation 
systems. In fact, only one of three pilot corporations developing an alternative evaluation 
system opted to design the entire system itself; the other two opted to contract with outside 
vendors for support on one or more parts of the evaluation system. 

The number of decisions needed to design an evaluation system can be overwhelming and 
requires significant time. Conversations regarding tools, systems, and processes that may seem 
concrete at the outset grow increasingly complex as more and more detailed questions are 
raised. One assistant superintendent cautioned, "I don't think you can underestimate the 
amount of time this takes. . .If districts have not started now, they will struggle next year."" 
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Finding the right combination of stakeholders to make 
adhering to a strict timeline can also be a challenge. 
Gathering input from those who will be part of the 
system is critical but time consuming. Many 
corporations form steering committees of teachers, 
principals and others to provide input on design 
decisions. Bringing together people with various 
opinions and interests can lead to increased buy-in, but 
requires additional time for each decision point. 
Corporations may struggle to meet deadlines if too 
many people, or those who aren't committed to 
solutions, are involved in the decision-making process. 


decisions or advise on decisions while 

/ ^ 
One assistant 
superintendent 
cautioned, "I don’t think 
you can underestimate 
the amount of time this 
takes..." 



Another challenge is developing the expertise needed to make design decisions. While there 
are many existing frameworks for classroom observation, few models with measures of student 
learning exist— a legally required component of all Indiana evaluation systems. While 4th-8th 
grade Language Arts and Math teachers have growth model data, the majority of teachers in a 
corporation do not have a statistical measure of student learning. Determining how to best 
measure student learning for these teachers can be difficult - this requires expert knowledge 
regarding assessments, which is often not available on many central office staff. 


Training and Communication 


1. Thorough, quality training is essential. 

Evaluators need training in order to develop the skills they need to implement a new evaluation 
system. In August, pilot evaluators were surveyed about their prior experiences with teacher 
evaluation. Only 52% reported having received training on observing and evaluating teachers 
from at least one source. Given the lack of training, over a third of evaluators were not 
confident in their ability to effectively evaluate teachers, provide effective instructional 
coaching, or help struggling teachers become more effective. ''' These are core skills of 
instructional leaders, and building administrators deserve to receive focused training to 
improve their practice. Therefore, one of the goals of implementing a new evaluation system in 
all corporations should be to ensure that instructional leaders have the tools and skills they 
need to confidently evaluate and give feedback to teachers. 

Amount and type of training in pilot corporations differed by corporation and evaluation 
system; however, most evaluators found it to be helpful. Evaluators felt that training improved 
their skills and provided a standardized way of looking at instruction, allowing them to focus 
on how students are performing.^ Pilot evaluators believed that training not only worked to 
build their understanding of complex components of the system like student learning measures, 
but also helped them to build core skills like taking evidence-based notes in observations, and 
effectively communicating the system to teachers.''' Improving the skills of evaluators also 
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helped to ensure successful implementation. Nearly eight in ten evaluators implemented key 
parts of the new evaluation systems with fidelity: using the observation mbric/ framework to 
guide classroom observations; taking detailed, evidence-based notes during observations; and 
discussing specific evidence with teachers after observations.’'” 

While pilot corporations had success with training, the complexity of planning for 
comprehensive training sessions should not be imderestimated. It is difficult to find time 


during summer months to train evaluators. 
Training during the school year usually requires 
evaluators to leave school buildings. 
Corporations may find it difficult to prioritize 
topics for training with the limited time given. 
While training has a positive effect on 
implementation, it can be challenging to know 
how much training to provide, given that training 
frequently takes evaluators away from other 
important aspects of their job. Although 
evaluators responded positively when asked 
about their training experiences, teachers still 





Training improved 
evaluators' skills and 
provided a 
standardized way of 
looking at instruction, 
allowing them to focus 
on how students are 



suggested in focus groups that administrators could have used more training in order to know 
and relay the system details correctly and consistently 

2. Frequent communication via various methods is critical to stakeholder understanding of 
the evaluation system. 

Teachers better understand the most important components of the evaluation system when they 
receive frequent communication from a variety of sources. 

Frequency of communication matters. Communicating with teachers about RISE at least once 
per week is associated with greater teacher understanding of the system.'’' However, it is 
essential that this communication include accurate, consistent, and applicable information. 
Teachers receive communication daily about many different school and corporation initiatives, 
and information that is not essential to practice is easily lost or forgotten. 

Corporations reported communicating about the new evaluation system using a number of 
methods: individual, department, and faculty-wide meetings, email updates, distributing hard 
copies of informational materials such as handbooks, and directing teachers to a website. With 
each additional method utilized both knowledge of the evaluation system, and opinion of the 
system as good for student learning, increases. Holding meetings and trainings with teachers, 
as well as providing informational resources, in hard copy or via website, were found to be 
especially helpful when communicating about more complex topics, like measures of student 
learning. 
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In focus group discussions, both teachers and evaluators noted the importance of clear and 
concrete communication.^* They pointed specifically to the importance of clearly defining and 
discussing new terminology for understanding and 
engaging with the new evaluation system. 

Communicating to stakeholders can present challenges. 

Corporations may struggle to know when it is too soon 
to reveal information that may lead to more questions 
than answers. Often, chains of communication are 
established so that corporation leadership 
communicates to administrators and administrators 
then communicate to teachers. This "whisper down the 
lane" approach is dangerous if it leads to diluted 
accuracy of information. Clear and consistent communication allows teachers to understand 
the new system, and this understanding leads to the tmst that is necessary for success. 

Initial Implementation 

1. Observation protocols that stress frequent observations and feedback using detailed, 
student-centered rubrics help to improve instruction and student learning outcomes. 

Student-centered rubrics focused on clear expectations for instruction are an essential part of 
improved observations across pilot corporations. 95% of evaluators say they use the 
observation rubric to guide classroom observations’**" and 85% of teachers report that their 
evaluators use the mbrics to help guide post-observation feedback discussions. ’**'* A strong 
majority of teachers also say that they use the rubric for planning lessons on a day-to-day basis 
(77%), to reflect on their instruction (82%), and to improve their practice (77 %).’*'* Qualitative 
research confirmed these findings; both teachers and evaluators think the rubric is a useful tool. 
As one RISE superintendent noted, "Probably every teacher would agree that the Teacher 
Effectiveness Rubric (captures) what they should be doing. You really can't argue with the 
rubric."’*'** 


r 




Clear and consistent 
communication allows 
teachers to understand 
the new system, and 
this understanding 
leads to trust needed 
for success. 






Improved observations with more opportunity for feedback lead to practices that improve 
student learning outcomes. Eight out of ten evaluators say that compared to last year, they are 
more confident that the feedback they provide helps make their faculty better teachers. ’*'*" 
Conversations between evaluators and teachers are now more frequent and based on concrete 
evidence. One superintendent stated, "Now, with the multiple observations, we have much 
more communication one-on-one. There is a lot more opportunity for feedback."’*'**** Almost all 
evaluators (99%) responded that their observations provide teachers the feedback they need to 
promote student learning. ***** 
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Like evaluators, teachers also believe the new evaluation systems increase opportunities to 
improve student learning. Teachers report they received more frequent feedback (65%) that 

was more relevant (61%) and helpful (61%) this 
year compared to prior years. Nearly seven in 
ten teachers agree that they consider their 
evaluator's feedback more frequently when 
planning daily lessons. They also agree that 
both observation feedback (79%) and the 
rubric/framework (68%) promote their students' 
learning. 

In RISE focus groups, teachers reported that 
post-observation feedback has been actionable 
and tied back to the rubric. Teachers expressed 
that they like when evaluators identify an area for improvement based on specific evidence. 
However, they also feel it is important that evaluators provide an opportunity to discuss what 
they might have missed during an observation. Feedback regarding teacher work on measures 
of student learning has also been valuable. According to one teacher, this type of feedback has 
resulted in more standards-driven teaching: " (Now), if (students) do not master a certain skill, 1 
will go back and teach them. We should be doing that. 

2. Lack of time to implement is a challenge for evaluators and teachers. 

Finding enough time to implement the new system was a challenge for many evaluators; 
however, most felt it was time well spent. In focus groups, school administrators and 
superintendents reported that evaluators were shifting their priorities from tasks they could 
delegate, like lunch duty, to evaluation. Still, a majority noted that they could use additional 
staff to complete some of the less critical aspects of their jobs so that they could focus more on 
evaluation.’'’^'' 

Many teachers also indicated that the biggest implementation challenge is lack of time.’'’"'’ In 
focus groups, teachers reported sacrificing planning time to focus on evaluation work. The 
most amount of time (4-6.5 hours) was spent working on the measures of student learning 
component of the evaluation system.’'’"'’’’ It should be noted that this time includes not only 
training teachers on measures of student learning, but also the time spent creating and 
improving assessments. Having high quality assessments can not only improve student 
outcomes, but the bulk of this work only has to happen in the first year of implementation. 
Teachers also expressed that their workload around measures of student learning could be 
made easier with clear communication about what historical and current student data is 
available and how best to access it. 


r 




One superintendent 
noted, "Probably every 
teacher would agree 
that tbe Teacher 
Effectiveness Rubric 
captures what they 
should be doing.” 
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Mid-Year Recommendations 


Based on the findings above, consider these recommendations when designing and planning for 
a new evaluation system. 

1) Begin planning and design work now. 

The time to start planning for next year is now. Whether designing your own system or 
adopting the RISE system, there are many crucial design decisions to be made at the 
corporation level including: 

• Who will be involved in researching and providing recommendations around system 
design? 

• Who will be involved in making key decisions about system design? 

• Who must approve the evaluation plan? 

• Who will be involved in measuring success of the system and refining the system 
accordingly? 

A steering committee of stakeholders should be created to provide diverse opinions and 
increase system-wide buy-in. Participants should represent key stakeholder groups in a 
corporation (central office administration, building level administration, a diverse set of 
teachers, etc.), and be mission-driven individuals who believe identifying teachers' strengths 
and areas for development will help improve teaching and learning in their corporation. 
Committee size should be large enough to represent various stakeholder groups, but small 
enough to make decisions efficiently— around 7-10 individuals as a guideline. Committee 
members must be committed to a timeline. Corporations should create a process for making 
decisions and moving forward if members of the committee cannot agree on a solution to a 
particular issue. 

The steering committee should familiarize itself with the requirements of the law as well as the 
DOE guidance on developing evaluation plans. This will help members to understand all of the 
factors and decisions involved in system design. If deciding whether or not to use parts or all of 
RISE, information related to this system can be found at www.riseindiana.org . 

Some (but not all) of the many design topics to address while writing an evaluation plan 
include: 

• Components of the Evaluation System - What will be involved in the evaluation of 
teachers other than a classroom observation protocol and measures of student learning? 
What will each of these components look like? 

• Scoring - How will each component of the system be scored? How will these scores 
form a summative score in one of the four performance categories? 
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• Training - What training is necessary for evaluators to learn about the components of 
the evaluation system? Will teachers also be trained? When will this training occur and 
what will it look like? 

• Professional Development - After observing teachers and identifying specific strengths 
and areas of development, what targeted opportunities will be provided to teachers to 
continually develop and improve their craft? 

• Remediation - What is the specific improvement plan for struggling teachers in danger 
of dismissal? 

• Data Collection and Plan Refinement - How will the corporation know if the 
evaluation system has been successful? What data needs to be collected and analyzed? 
What is the process for improving the evaluation plan based on data analysis and 
feedback? 

A more comprehensive list of topics to be included in an evaluation plan can be found on the 
DOE guidance website. 

A work plan should include time for research (if using a system other than RISE), writing 
multiple drafts of the plan, approval (if necessary), frequent communication with stakeholders, 
and training prior to implementation. If a corporation leaves out any of these crucial 
components, staff will not have enough time to plan for fall implementation. 

Eor efficiency, consider dividing the steering committee into smaller groups to focus on the 
various design topics listed above. These smaller groups can perform the necessary research 
and then provide recommendations to the larger group. Ultimately, be creative, but don't 
reinvent the wheel. There is a multitude of existing evaluation systems other than RISE. Two 
alternative pilot districts are using pre-existing systems for their observation protocol and 
training (for more information on these systems, see Appendix A-2). Take time doing research 
on RISE and other systems before deciding to start from scratch. 


2) Focus system design on a high-touch, low-inference observation protocol and accurate, 
fair measures of student learning. 

Each evaluation system will have different components that form a summative evaluation 
rating. All, however, must include some type of classroom observation protocol, and measures 
of student learning per Indiana law. 
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When designing an observation protocol, there are a few best practices to keep in mind: 

• DesigiVpurchase a low-inference observation rubric focused on students. Low- 
inference language requires little subjectivity on the part of the evaluator. Expectations 
for good instruction should be clearly laid out, with examples whenever possible. Avoid 
language that allows different interpretations. The most common mistake here is 
differentiating performance levels based on vague frequency terms such as 
"sometimes", "often", "occasionally", etc. If subjective language cannot be avoided, 
training must bring evaluators to a common understanding of terms. Rubrics should 
also be focused on student actions instead of teacher actions whenever possible. 
Evaluation of teacher performance should be based on whether or not students are 
learning. Student engagement, interactions with each other, and interactions with the 
teacher all show student learning. Many rubrics make the mistake of only looking at 
teacher actions, without observing students. 

• Design a process that accounts for regular, timely, and actionable feedback. Every 
observation should be followed by either written or verbal feedback. This feedback must 
occur in a timely manner following the observation. In order for it to be helpful, 
feedback must provide concrete evidence of what was observed in the classroom and 
specific suggestions for what can be improved in the future. Eeedback should be directly 
aligned to the observation rubric. Suggestions for professional development 
opportunities aligned with areas for improvement are particularly helpful. 

• Include multiple observations of sufficient length to monitor progress throughout the 
year. One of the primary goals of a good evaluation system should be to develop 
teachers over time. Observing a teacher once a year will not allow the teacher to 
incorporate feedback and show progress over the course of the year. Indiana law 
specifies that teachers be observed a minimum of two times per year. In reality, the 
strongest observation protocols include many more observations. There are many 
combinations of short and long or announced and unannounced observations that lead 
to a successful evaluation system. The important thing is that the evaluator is able to get 
an accurate picture of teacher performance over time and provide frequent feedback to 
teachers on their performance. 

When developing measures of student learning for an evaluation system, best practices to 
consider are to: 

• Start by identifying good assessments of student learning. The first step in designing 
good measures of student learning is ensuring that all teachers, regardless of their 
subject or grade level, have good assessments of student learning. The state provides 
assessments for some grades and subjects (ISTEP, ECA, LAS Links, etc), but for many 
others, corporations and school leaders will be responsible for identifying assessments. 
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The assessment matrix tool on the RISE website will help organize information around 
assessments, regardless of whether or not a corporation is using RISE. If a corporation or 
school decides to create common assessments where none currently exist, collaboration 
between teachers is essential for creating a good product. 

• Take care not to use false measures of precision. The concept of student growth or 
progress can be complicated. While the state provides a statistical measure for growth in 
some grades and subjects (ELA and Math in grades 4-8), many teachers are not covered 
by this measure. Be careful about using a pre-test and post-test method. A common 
mistake is subtracting scores on tests between the begirming and end of year. If the pre- 
and post-test are not aligned in terms of rigor and content, this is not an accurate 
measure of growth. Eurthermore, corporations should avoid testing students on content 
they have not yet learned. Rather than trying to devise complex calculations with little 
assessment expertise, try to think about student progress in broader terms. How do 
teachers in your district know if students have learned? Do they set goals for 
themselves? Do they have assessments that measure this progress? Begin your design 
process by gathering this type of information from teachers. 

• Use the resources available. The DOE provides guidance around assessments and 
rigorous measures of student learning on the DOE Guidance website. To see what 
measures are being used in RISE, visit the Measures of Student Learning page on the 
RISE website . 

3) Prioritize transparent, accurate and frequent communication with stakeholders, 

underscoring the link between teacher development and improved student learning. 

The ultimate goal of an effective evaluation system is to develop teachers and improve student 
outcomes. Eor implementation to be successful, teachers and administrators need to be clear 
about their specific roles and responsibilities, including how the new process will ultimately 
improve their practice. They also need to feel confident that their questions will be answered. 
This requires frequent two-way communication through multiple channels and media, 
including accurate and detailed information about the components of the system, as well as 
answers to common questions. Most importantly, communications should imderscore the link 
between teachers' classroom practice and student learning. 

The following are best practices when communicating to stakeholders: 

• Communication should take place early and often in order to provide transparency 
and increase stakeholder buy-in prior to implementation. Reach out to administrators 
and teachers with all available information, and fill in the details as they come. 
Corporations or administrators may think that they are doing their staff a favor by 
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waiting until all details of the system are resolved before communicating, but this 
approach leaves staff in the dark, and may result in a negative perception of the system. 

• Offer a variety of types of communication. Different people learn best through 
different channels and media. Communicate using a variety of methods such as 
individual, department, and faculty meetings, email updates, and hard copies of 
resources. The more methods you use to communicate the information, the better. 

• Communicate to all stakeholders. Prioritize commimication with evaluators and 
teachers, but don't forget about other key stakeholders. Many corporations will need to 
present information to school boards, and may also wish to communicate with parents 
and community members about the new system. 

• Remember to frame communication appropriately for the audience. Different sets of 
stakeholders have differing sets of priorities. Communication may need to be re-framed 
for a school board, for parents, or for teachers. No matter the audience, make sure the 
information is digestible and framed to show how the new evaluation system 
compliments the many other efforts teachers and schools are already undertaking to 
advance student learning. 

4) Allocate sufficient time and resources for training. 

If opting to use part or all of the RISE system. Educational Service Centers (ESC) are providing 
monthly trainings on different topics between Eebruary and August 2012, with an intensive 
four day training during the summer. Contact your local ESC for more information on RISE 
training. 

If using a system other than RISE, consider how training will be supported. Some corporations 
opting to partner with an established system (such as TAP) will receive training as part of their 
contract. If developing a imique system, corporations may still consider bringing in outside 
consultants to design training. While the least expensive option might be corporation-designed 
trainings, this requires a significant amount of time and expertise to do well. RISE pilot 
corporations will receive a total of 16-20 hours of training this year. Use this time estimate to 
help plan for a training schedule and costs. 

If planning and designing training at the corporation level, consider the following best 
practices: 

• Choose topics early and start planning as soon as design is complete. A frequent 
mistake is to try and start planning training before the details of the system have been 
finalized. This typically occurs when decisions are delayed and the design timeline has 
been extended. Avoid this. Appropriate time to plan training should be protected in the 


13 


work plan. Ensure that the training plan covers all necessary components of the 
evaluation system (observation protocol, measures of student learning, giving feedback, 
scoring, etc). 

• Organize training topics to prioritize those that require work prior to system 
implementation. Any initial training should include a broad overview of the system. 
After this, however, plan to train first on topics that may require early work or practice. 
For example, if prepared to do so, provide training on the observation rubric during the 
current school year while administrators have a chance to practice in classrooms. If 
assessments of student learning need to be developed prior to system implementation, 
plan for this training and work accordingly. 

• Design training to be as hands-on as possible. Incorporate small group discussion, 
debate, role-playing and workshops. Adults learn more when they are actively engaged 
in the learning process. Minimize the time spent lecturing and maximize time for 
practice and activities. 

• Deliver training in digestible sessions. Topics that require teaching new skills or 
processes can be overwhelming. While many educators are familiar with observation 
rubrics and protocols, other parts of the system, such as measuring student learning, 
may be brand new. For these types of topics, hold several training sessions rather than 
one long training session to allow time for participants to digest information. Every 
session should include time to review content from the previous section and reflect on 
any questions or concerns. 

Training for staff will differ depending on their level of involvement as evaluators. In most 
corporations, evaluators will include school-based administrators, but some corporations may 
also opt to include other individuals (teacher leaders, central office administrators, outside 
hired evaluators, etc.). All individuals who contribute to evidence collection surrounding 
teacher evaluation must be trained on the evaluation system per law. When it comes to 
classroom observations, multiple visits by multiple evaluators is preferable. It leads to more 
reliable ratings and provides teachers with a wider range of real-time feedback on their practice. 
At the same time, it is important to remember that all evaluators need to be carefully trained to 
ensure that teachers have confidence in their observers' abilities to effectively respond to what 
they see in the classroom. Certain components of a new evaluation system may also require 
additional work or sldll-building for teachers (introduction to a new rubric, designing 
assessments for measuring student learning, etc.). 

When teacher training is included, consider whether administrators train teachers in their 
buildings or if this training will be offered centrally at the corporation level. Regardless of 
whether or not teachers need to be trained on components of the system, it is essential that the 
entire system be communicated clearly and thoroughly. 
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5) Help administrators and teachers think about time management in advance of system 

implementation. 

The additional requirements of a new evaluation system will necessitate that administrators 
allocate more time in their daily schedules for classroom observations and conferencing than 
ever before. To do this, there will be parts of their current routine that are given less time or 
eliminated. Administrators will need to re-focus their time on activities that allow them to be 
strong instructional leaders in their schools. 

Consider the following strategies for assisting administrators with time management: 

• Offer professional development sessions or workshops on time management 
strategies this year. If the evaluation system splits responsibilities between principals 
and assistant principals (or even others in the building), have these individuals work 
together to assess the work they do on a daily basis and plan for ways they can work 
more efficiently and effectively starting next year. If there are certain increased time 
commitments for principals that others will not have, help principals consider how they 
can better delegate non-evaluation responsibilities with their staff. 

• Re-evaluate central office demands of administrators. Central office staff should take 
the time to review all requests made of administrators, in order to find ways of reducing 
paperwork or other redundancies in the system. Ensure that requests are necessary so 
that administrators can spend the time they need as instructional leaders in their 
schools. Job descriptions and hiring for the corporation should also reflect the 
importance of administrators as instructional leaders. 

• Offer efficient ways of collecting and storing data, A significant amount of time in a 
new evaluation system is dedicated to completing necessary paperwork to track 
information on teacher performance. Ensure that the forms included in the evaluation 
system are all necessary and not repetitive in terms of information collected. If 
information eventually needs to be submitted electronically, look for solutions where 
evaluators can enter data in real time during the observation rather than transfer 
information from paper to computer later. Most pilot corporations are currently using 
or looking for electronic solutions to assist with this. Those using these solutions have 
found that they can save a lot time both during and after an observation. 

As with any new skill, there is a learning curve with the work associated with new evaluation 
systems. Eor example, many RISE evaluators initially spent more than an hour to organize 
observation notes and prepare rubric -aligned feedback after an observation. However, after 
becoming more familiar with the process, the rubric, and assisting technology, this activity now 
takes about half the time it used to. 
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School leaders can also help teachers with effective time management and help save time by 
providing the resources needed to do the work. Ensure that teachers have easy access to 
current and historical student learning data at all times and provide them with the time they 
need for collaboration and independent work that may be required of them with the new 
evaluation system. 

Conclusion 

No matter what system is adopted or what training takes place, evaluation system design, 
planning, and implementation requires a serious time commitment and a willingness to 
transform school culture. For many educators, the new process represents a notable change in 
the way they spend their time and energy. In most 
schools, it will also require a deliberate shift in the way 
educators approach the relationship between teachers 
and administrators. 

By emphasizing collaboration, classroom visits, real-time 
feedback, and frequent conversations about instruction 
and student learning, a new evaluation system gives 
school leaders the process they need to transcend the role 
of building manager and become true instructional 
leaders, engaged in the classroom, developing their 
teachers. 

It's hard work, but it's the right work. Corporations who 
dedicate appropriate time to the development and implementation of a good evaluation system 
will begin to notice a shift in culture within schools. In just a few months, teachers and 
evaluators in pilot corporations have reported having more conversations about classroom 
practice than ever before, helping everyone focus their energies on our most important goal: 
helping students learn. 


r 






It gives school leaders 
the process they need 
to transcend the role of 
building manager and 
become true 
instructional leaders, 
engaged in the 
classroom, developing 
their teachers. 
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Appendix A-1 

Pilot Corporation Profiles 


Corp 

ID 

Corp Name 

Total 

Enrollment 

(1112) 

American 

Indian 

Black 

Asian 

Hispanic 

White 

Multiracial 

FRL 

Pass 

ELA& 

Math 

ISTEP+ 

(10-11) 

Pass ELA 

& Math 
ECA (lo- 
ll) 

Grad 

Rate 

(2011) 

0235 

Fort Wayne Community 
Schools 

30821 

0.6% 

24.7% 

4.2% 

13.8% 

49.1% 

7.6% 

69.2% 

63.0% 

49.7% 

88.1 

2920 

Bloomfield School District 

1035 

0.3% 

0.7% 

0.6% 

2.4% 

93.7% 

2.3% 

37.5% 

69.4% 

70.9% 

97.2 

1730 

Greensburg Community 
Schools 

2294 

0.3% 

0.0% 

0.8% 

1.7% 

95.2% 

2.0% 

50.7% 

79.4% 

65.9% 

96.4 

5360 

MSD Warren Township 

11899 

0.2% 

46.2% 

1.0% 

10.0% 

35.6% 

7.0% 

66.5% 

62.4% 

54.7% 

87.6 

5380 

Beech G rove City Schools 

2734 

1.0% 

7.3% 

0.7% 

5.5% 

79.6% 

6.0% 

63.6% 

72.5% 

67.5% 

84.0 

5480 

Bremen Public Schools 

1461 

0.0% 

0.3% 

0.6% 

21.4% 

76.1% 

1.5% 

39.1% 

79.2% 

75.2% 

87.8 


Notes: 


• The passing percentages provided for ECA are for first-time test takers. 

• Enrollment data are from SY 11-12, while ISTEP+, ECA and grad rates are from 2010-11; 
these are the most recent data available for each data element. 


Appendix A-2: Descriptions of Pilot Evaluation Systems 

RISE (Bloomfield, Fort Wayne, Greensburg) 

RISE consists of two main components. Professional Practice and Student Learning Measures, 
which are combined to determine a teacher's summative rating. 

The Professional Practice component is measured by the Indiana Teacher Effectiveness Rubric, 
which consists of four domains: Planning, Instruction, Leadership, and Core Professionalism. 
Evidence of a teacher's Professional Practice is collected during a minimum of two formal forty- 
minute observations and three short twenty-minute observations, although some pilot 
corporations choose to do more. All observations must be followed by written feedback, and 
long observations are also followed by required post conferences. Observations are performed 
by primary and secondary evaluators. All pilot corporations are utilizing building 
administrators only as primary and secondary evaluators this year. 

Student Learning Measures are comprised of three pieces of data: Individual Teacher Growth 
Model based on ISTEP scores (Grades 4-8 ELA and Math), School Wide Learning based on the 
A-E Accountability Policy, and Student Learning Objectives, teacher-written goals around 
rigorous assessments. 

A teacher receives a 1-4 rating for Professional Practice and each of the three Student Learning 
Measures. Those ratings are rolled into an overall summative rating. Depending on a teacher's 
mix of classes. Professional Practice comprises 50-75% of a teacheP s summative rating, and 
Student Learning Measures comprise the remaining 25-50%. 
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For more information about RISE, please visit www.riseindiana.org 


TAP System (Beech Grove) 

The evaluation component of the TAP System considers both classroom lesson evaluations and 
student achievement growth measures. The classroom lesson evaluations are measured using 
the TAP rubric which includes four domains: Designing and Planning Instruction, Instruction, 
The Learning Environment, and Responsibilities. Those four domains are further defined by 19 
areas of effective instructional practice and an annual survey leading to an annual overall 
"Skills, Knowledge, and Responsibilities" (SKR) score. The TAP System requires 4-6 formal 
evaluations each year, two announced and two unannounced. Announced formal evaluations 
include a pre-conference and all four formal evaluations have post-conference and self- 
evaluation components. Observations are performed by master teachers, mentor teachers, and 
one or more administrators throughout the year. 

Student achievement growth measures include both individual teacher/classroom-level data 
(when available) and school-level data. When determining both differentiated levels of teacher 
effectiveness and also performance-based compensation amounts, 50% of a teacher's rating is 
based on their annual SKR score and 50% is based on student achievement growth measures. 

More information on the TAP evaluation system can be found at: www.tapsystem.org 

Bremen Evaluation System (Bremen) 

Bremen's evaluation system consists of two main components, an assessment of a teacher's 
Professional Skills and Measures of Student Learning, which combine to determine a teacher's 
summative rating. 

Professional Skills are measured by the Bremen/ McREL teacher evaluation rubric which 
consists of five standards: Leadership, Respect, Knowledge of Content Taught, Instruction, and 
Reflection on Practice. Evidence of a teacher's Professional Skills is collected during a 
minimum of three or four forty-minute observations. A pre-conference is required prior to the 
first formal observation, but not required for future formal observations. A post-conference is 
required following each formal observation. Observations are performed by building 
administrators. 

Measures of Student learning are comprised of four pieces of data: school-wide growth based 
on ISTEP and ECA assessments, school-wide achievement based on ISTEP and ECA 
assessments, individual growth based on locally created assessments, and individual 
achievement based on locally created assessments. 

A teacher receives an overall rating for Professional Skills and an overall rating for Measures of 
Student Learning. Those ratings are rolled up into an overall summative rating. Professional 
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Skills comprise 75 % of the summative rating, and Measures of Student Learning comprise 25 % 
of the summative rating. 

MSD Warren Evaluation System (MSD Warren) 

MSD Warren is in the process of finalizing the design of its teacher evaluation system. It 
contains two main components, a measure of a teached s Professional Practice and Measures of 
Student Learning, which combine to determine a teacher's summative rating. 

The Professional Practice component is measured by a teacher effectiveness mbric still in 
development. Currently, the rubric consists of four domains: Instmctional Planning, Effective 
Instruction, Classroom Environment, and Professional Commitment. Evidence of a teacher's 
Professional Practice is collected during a minimum of four long, class-length observations and 
eighteen short five to seven minute observations. One long observation is announced, with a 
pre and post conference. Three long observations are unannounced and followed by written 
feedback. Short observations are followed by written feedback. Observations are performed by 
building administrators. 

Measures of Student Learning are comprised of two pieces of data: Individual Growth Data and 
School-wide Growth Data. The means by which these two pieces will be measured is still in 
development. 

A teacher receives a rating for each of the three individual rubric domains and the two 
Measures of Student Learning. Those ratings are rolled into an overall summative rating. 
Professional Practice comprises 70% of a teacheP s summative rating, and Measures of Student 
Learning comprise the remaining 30%. 


Comparison of Systems 



RISE (Bloomfield, Fort 
Wayne, Greensburg) 

TAP (Beech Grove) 

Bremen 

MSD Warren 

Minimum Observations 

2 40-minute observations 

3 10-minute observations 

*ln addition to the RISE 
minimum requirements, 

Fort Wayne Community 
Schools requires daily 1-3 
minute snapshots 

4 formal observations (2 
announced, 2 
unannounced) 

3-4 40-minute 

observations 

4 class-length 
observations (1 
announced, 3 
unannounced) 

1 5-7minute observation 
once every three weeks 

Observation Rubric used 

Indiana Teacher 

Effectiveness Rubric 

TAP Rubric 

Bremen/McREL Rubric 
• Leadership (17.5% of 

District revised Teacher 

Effectiveness Rubric 


• Planning (10% of 

Designing and Planning 

rubric score) 

• Instructional 


rubric score) 

Instruction (15% of rubric 

• Respect (17.5% of 

Planning (10% of 


• Instruction (75% of 

score) 

rubric score) 

summative score) 


rubric score) 

Learning Environment (5% 

• Knowledge of 

• Effective Instruction 


• Leadership ( 15% of 

of rubric score) 

Content Taught 

(50% of summative 


score) 

Instruction (75% of rubric 

(17.5% of rubric 

score) 


• Core Professionalism 

score 

score) 

• Classroom 


(factored after other 

Responsibilities (5% of 

• Instruction (30% of 

Environment (10% 


three domains are 
rolled up) 

rubric score) 

rubric score) 

• Reflection on 

Practice (17.5% of 
rubric score) 

of summative score) 

• Professional 

Commitment 
(factored after other 
three domains) 
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‘Overall rubric score 
calculated before being 
weighted into summative 

‘These weightings are 
different for TAP Master 

and Mentor teachers to 
reflect their specific job 
responsibilities. 

‘Overall rubric score 
calculated before being 
weighted into summative 

“Overall rubric score not 
calculated before being 
weighted into summative 

student Data used 

Individual Growth Model 
Data (Grades 4-8 ELA and 
Math) (20-35% of 
summative score) 

School Wide Growth 

Measure - based on A-F 
accountability policy (5% 
of summative score) 

Individual Growth Model 
Data (Grades 4-8 ELA and 
Math) (30% of summative 
evaluation where 
available) 

School Wide Growth 
Measure (20-50% of 
summative evaluation) 

School-wide growth (15% 
of data score) 

School-wide achievement 
(15% of data score) 

Individual teacher student 
growth (30% of date 
score) 

School-wide growth (10% 
of summative score) 

Individual teacher student 
growth (20% of 
summative score) 


Individual Student 

Learning Objectives (10- 
20% of summative score) 


Individual teacher student 
achievement (30% of data 
score) 



‘Overall student learning 
not score calculated 
before being weighted 
into summative. 


“Overall student 
learning score calculated 
before being weighted 
into summative. 

‘Overall student learning 
score not calculated 
before being weighted 
into summative. 

% of Summative 

Evaluation based on 

Rubric Score 

50-75% depending on a 
teacher's classes 

50% 

75% 

70% 

% of Summative 

Evaluation based on 

Student Data 

25-50% depending on a 
teacher's classes 

50% 

25% 

30% 


Endnotes 


' TNTP, a national nonprofit organization founded by teachers, works with schools, districts and states to advance 
policies and practices that ensure effective teaching in every classroom. TNTP partnered with the IDOE to provide 
direct support to the three pilot corporations implementing RISE this year. The findings in this report are based on 
TNTP's analysis of data collected from all six pilot corporations. Recommendations are based not only on Indiana 
pilot findings, but on TNTP's experience doing similar work with other states and districts nationwide. 

" Source: Focus Groups and individual conversations at Pilot Corporations. 

52% of evaluators responded strongly agree or agree to receiving training on how to observe and/or formally 
evaluate teachers from at least one of five different sources prior to June 2011. (n=88.) Source: Indiana Pilot 
Corporations Beginning of Year Evaluator Survey, administered September-October 2011. 

''' 38%, 37%, and 35% of evaluators responded without confidence (somewhat agree, somewhat disagree, disagree, 
or strongly disagree) respectively to the statements "I provided effective instructional coaching and strategies 
(n=95)," "I effectively addressed poor instructional performance by helping struggling teachers become more 
effective (n=95)," and "I accurately evaluated all teachers (n=95)." Source: Indiana Pilot Corporations Beginning of 
Year Evaluator Survey, administered September-October 2011. 

'' Source: Focus Groups and individual conversations at Pilot Corporations. 

''' 66%, 67%, and 66% of evaluators responded strongly agree or agree respectively when asked their level of 
agreement with the statements "I received training that helped me take good observation notes when observing 
teachers (n=154)," "I received training that helped me effectively communicate the new teacher evaluation system 
to teachers (n=153)," and "I received training that helped me understand how student learning (achievement 
and/or growth) is factored into the new teacher evaluation system (n=134)." Source: Indiana Pilot Corporations 
Mid-Year Evaluator Survey, administered 1/16/12-1/27/12. 
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85%, 78%, and 78% of evaluators responded strongly agree or agree respectively when asked their level of 
agreement with the statements "I use the new system's observation rubric/framework to guide my classroom 
observations (n=133)," "I take detailed, evidence-based notes of what I see in my teachers' classroom (n=142)," 
and "After performing an observation, I discuss specific evidence of what I observed in the classroom with the 
teacher (n=142)." Source: Indiana Pilot Corporations Mid-Year Evaluator Survey, administered 1/16/12-1/27/12. 

Source: Focus Groups and individual conversations at Pilot Corporations. 

After controlling for communication method, teachers who received communication about the new evaluation 
system at least once per week were 50% more likely to be considered to have the highest knowledge of the system 
than those who received less frequent communication. Teachers' knowledge was measured by their responses to 
14 true/false questions about the new evaluation system. Those considered to have the highest knowledge were 
those teachers who answered enough questions correctly to put them in the top quartile of teachers. Knowledge 
was only asked of RISE teachers. Source: Indiana Pilot Corporations Mid-Year Teacher Survey, administered 
1/16/12-1/27/12. 

Controlling for frequency, every additional method of communication used increases both knowledge and 
perception of the system as good for students by .3 on a 10 point scale. See above footnote for actual knowledge 
variable explanation. Perception of the system as good for student learning is a composite variable with a 10 point 
scale created from 4 questions about whether the system is good for student learning. Source: Indiana Pilot 
Corporations Mid-Year Teacher Survey, administered 1/16/12-1/27/12. 

Controlling for frequency, holding meetings to train teachers on the student learning measures process, directing 
teachers to a website about the new evaluation system, meeting with department or grade-level teams to discuss 
the student learning measures process, and providing a hard or electronic copy of the student learning measures 
process respectively increased actual knowledge by .5, .4, .3, and .3 on a 10 point scale. See above footnote for 
actual knowledge composite variable explanation. Source: Indiana Pilot Corporations Mid-Year Teacher Survey, 
administered 1/16/12-1/27/12. 

Source: Focus Groups and individual conversations at Pilot Corporations. 

95% of evaluators responded net agree (strongly agree, agree, somewhat agree) when asked their level of 
agreement with the statement "I use the new system's observation rubric/framework to guide my classroom 
observations." (n=133.) Source: Indiana Pilot Corporations Mid-Year Evaluator Survey, administered 1/16/12- 
1/27/12. 

85% of teachers responded net agree (strongly agree, agree, somewhat agree) when asked their level of 
agreement with the statement "After being observed, my evaluator uses the new system's observation 
rubric/framework to discuss what he or she observed in the classroom with me." (n=1619.) Source: Indiana Pilot 
Corporations Mid-Year Teacher Survey, administered 1/16/12-1/27/12. 

77%, 82%, and 77% of teachers responded net agree (strongly agree, agree, somewhat agree) respectively when 
asked their level of agreement with the statements "I consider the new teacher evaluation system's observation 
rubric/framework when planning lessons on a day-to-day basis (n=1650)," "I often reflect on my own instruction 
and consider how it fits within the new teacher evaluation system's observation rubric/framework(n=1651)," and 
"I consider the new teacher evaluation system's observation rubric/framework when seeking ways to improve my 
practice(n=1648)." Source: Indiana Pilot Corporations Mid-Year Teacher Survey, administered 1/16/12-1/27/12. 

Source: Focus Groups and individual conversations at Pilot Corporations. 

86% of evaluators responded net agree (strongly agree, agree, somewhat agree) when asked their level of 
agreement with the statement "Compared to last year, I am more confident that the feedback I provide will help 
make the faculty better teachers." (n=132.) Source: Indiana Pilot Corporations Mid-Year Evaluator Survey, 
administered 1/16/12-1/27/12. 

Source: Focus Groups and individual conversations at Pilot Corporations. 

99% of evaluators responded net agree (strongly agree, agree, somewhat agree) when asked their level of 
agreement with the statement "My observations of teachers provide teachers the feedback they need to promote 
their students' learning." (n=146.) Source: Indiana Pilot Corporations Mid-Year Evaluator Survey, administered 
1/16/12-1/27/12. 

65%, 61%, 67%, and 61% of teachers responded net agree (strongly agree, agree, somewhat agree) respectively 
when asked their level of agreement with the statements "Compared to the last time I was evaluated, my 
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evaluator has provided more feedback that helps me improve my instruction (n=1575)/' my evaluator has 
provided me feedback that is more relevant to the classes and students I teach (n=1573)/' more frequently 
consider my evaluator's feedback when planning daily lessons (n=1573)/' and I am more confident that my 
evaluator's feedback will help me become a better teacher (n=1566)." Source: Indiana Pilot Corporations Mid-Year 
Teacher Survey, administered 1/16/12-1/27/12. 

79% and 68% of teachers responded net agree (strongly agree, agree, somewhat agree) respectively when asked 
their level of agreement with the statements "My evaluator's observation of my instruction gives me the feedback 
I need to improve my students' learning (n=1595)," and "Having access to the RISE observation rubric/framework 
guides my instruction in a way that will promote my students' learning (n=1597)." Source: Indiana Pilot 
Corporations Mid-Year Teacher Survey, administered 1/16/12-1/27/12. 

Source: Focus Groups and individual conversations at Pilot Corporations. 

60% of evaluators responded that "there is not enough time to implement many of the requirements of the 
new teacher evaluation system" as causing the biggest challenge to implementing the new teacher evaluation 
system (forced choice). (n=145.) 69% of evaluators responded net disagree (strongly disagree, disagree, somewhat 
disagree) when asked their level of agreement with the statement "The time I've spent implementing the new 
teacher evaluation system could have been better spent on other aspects of my job." (n=143.) Source: Indiana 
Pilot Corporations Mid-Year Evaluator Survey, administered 1/16/12-1/27/12. 

Source: Focus Groups and individual conversations at Pilot Corporations. 

60% of evaluators responded net agree (strongly agree, agree, somewhat agree) when asked their level of 
agreement with the statement "I need(ed) additional staff to complete some of the non-evaluation aspects of my 
job." (n=144.) Source: Indiana Pilot Corporations Mid-Year Evaluator Survey, administered 1/16/12-1/27/12. 
xx''| 35 % gf teachers responded that "there is not enough time to implement many of the requirements of the new 
teacher evaluation system" as causing the biggest challenge to implementing the new teacher evaluation system 
(forced choice). (n=1659.) Source: Indiana Pilot Corporations Mid-Year Teacher Survey, administered 1/16/12- 
1/27/12. 

Source: Focus Groups and individual conversations at Pilot Corporations. 

Non-RISE Teachers spent a median of 4 hours on measuring student learning; RISE Teachers spent a cumulative 
median of 6.5 hours on measuring student learning. Source: Indiana Pilot Corporations Mid-Year Teacher Survey, 
administered 1/16/12-1/27/12. 


22 



